DOCUMENT RESUME 



ED 456 309 



CE 082 289 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



AVAILABLE FROM 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Sanderson, Kay; Pearce, Ron 

Performance Level Assessment: Developing Quality and 
Consistency through Research Partnerships. 

2001-03-00 

9p . ; In: Research to Reality: Putting VET Research To Work. 
Proceedings of the Australian Vocational Education and 
Training Research Association (AVETRA) Conference (4th, 
Adelaide, Australia, March 28-30, 2001); see CE 082 232. 

For full text: 

http : / /www . ave t ra . org . au/PAPERS%2 02 0 0 l/Sanderson%2 0 &%2 0 Pearc 
e .pdf . 

Reports - Descriptive (141) -- Speeches /Meeting Papers (150) 

MF01/PC01 Plus Postage. 

Competence; Competency Based Education; Cooperative 
Planning; *Criterion Referenced Tests; Definitions; 
Educational Cooperation; Educational Quality; Educational 
Research; Evaluation Criteria; Evaluation Methods; Foreign 
Countries; Hospitality Occupations; * Partnerships in 
Education; *Performance Based Assessment; Postsecondary 
Education; *Research and Development; Student Certification; 
★Student Evaluation; Systems Approach; Teamwork; *Vocational 
Education 

Australia (Queensland) ; *Hospitality Education; Performance 
Levels; *TAFE (Australia) 



ABSTRACT 



Performance level assessment (PLA) is a type of 
supplementary evidence system for competency-based assessment that is being 
pilot tested at five technical and further education (TAFE) institutes 
throughout Queensland, Australia. The PLA system uses the following three 
levels of performance: competency achieved; competency achieved with credit; 
and competency achieved with distinction. To earn ratings of competency 
achieved with credit and competency achieved with credit, students must meet 
six and seven specific criteria, respectively. During the pilot project, 

North Point Institute of TAFE (NPIT) and Cooloola Sunshine Institute of TAFE 
(CSIT) formed a research partnership to collectively improve their approach 
to implementation of PLA. Hospitality teachers in each institute met and 
developed objective criteria against a series of performance criteria. 
Additional meetings were held to translate the criteria to other assessment 
areas, design a template stating the agreed criteria, and discuss the 
following issues: (1) the fact that the objective criteria were open to 

subjective interpretation; (2) the use of performance indicators; and (3) the 
adequacy of the previously agreed assessment criteria. Through their 
partnership arrangement, teachers from NPIT and CSIT were able to conduct 
ongoing research and review aimed at improving the quality of Queensland's 



new PLA system. (MN) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



Performance Level Assessment: developing quality and 
consistency through research partnerships 

Kay Sanderson and Ron Pearce 



Performance Level Assessment (PLA) is a type of supplementary evidence system 
for competency -based assessment being trialled by Technical and Further Education 
(TAFE) Queensland. It was initiated in 1999 based on trials of a supplementary 
evidence system undertaken by the Architectural Technology Department of South 
Bank Institute of TAFE (SBIT) in 1999. 

The Morningside campus of SBIT obtained approval from the Institute Director and 
gained support from employers and their Industry Association to trial this system. 
The main reason for the trial was to find a way that students wishing to articulate to 
university would not be disadvantaged. The outcomes of the trial were positive. 

After investigating the outcomes from the trial, in January 2000 the Board of TAFE 
Queensland, through the Institute Reference Group (IRG) for Product, commissioned 
a project on PLA. The term PLA was chosen to avoid any confusion with grading 
systems based on the provision that units of competency cannot be graded. It was 
anticipated it would reduce the number of complaints received by TAFE Queensland 
from students, parents, employers and industry groups by issuing a supplementary 
evidence report to students based on their performance levels. The IRG (Product) set 
up a working party to provide guidance and support to the project and 
commissioned the Centre for Advancement of Innovative Learning (CAIL), Wide 
Bay Institute of TAFE, to manage the project. Endorsement for the project has not 
been sought from the State Training Authority as yet, as it is currently a TAFE 
response to client needs. Also, it is recognised that the PLA system is likely to be an 
interim arrangement for TAFE Queensland and is likely to provide an informed 
position to state, other states, the Australian National Training Authority (ANTA) 
and national inquiries concerning supplementary evidence systems. 



How does PLA work? 

The PLA system uses three levels of performance: competency achieved, competency 
achieved with credit, and competency achieved with distinction. Initially students 
are assessed as competent or not yet competent based on assessment against a unit of 
competency. A student must be assessed as competent before being considered for a 
PLA of credit or distinction. To be assessed as competent with credit, a student must: 






demonstrate advanced reliability and responsibility; 

provide evidence of being able to access additional, relevant information and 
apply appropriately; 

present work with originality and/ or creativity; 



demonstrate positive approaches to learning 

demonstrate the ability to work and learn independently and/ or within a 



team; and 
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• provide evidence of planning and resource allocation to ensure completion of 
tasks. 

To be assessed as competent with distinction, a student must: 

demonstrate exceptional reliability and responsibility; 
provide significant evidence of being able to acquire, validate and apply 
relevant additional resources and sources of information and use them 
effectively and efficiently; 

demonstrate significant work output, quality, originality and creativity; 
demonstrate positive approaches to learning with a high degree of 
motivation; 

display a significant level of independence; 

provide evidence of monitoring and evaluation of work, including the ability 
to identify opportunities for improvement; and 

demonstrate mastery in application of tasks, ensuring flexibility, efficiency 
and effectiveness. 

In some cases it may not be possible to assess a student against all the criteria for 
each level of credit or distinction. Teachers are required to select those most relevant 
to the unit of competency or cluster of units of competency that they are assessing. 
Performance indicators were used in the trials and are an option to assist teachers in 
establishing an overall credit or distinction level of performance for each student. 

The process allows for teachers to give feedback to students on their progression and 
is intended to provide motivation towards achievement of an overall performance 
level. Performance indicators range from one to five, each level indicating the 
following: 

1. Minimal evidence of relevant criteria demonstrated 

2. Some evidence of relevant criteria demonstrated 

3. All relevant criteria demonstrated to achieve competency 

4. Advanced level of performance demonstrated 

5. Exceptional level of performance demonstrated. 

Scope of the project 

It is intended that the scope of the project will be such that it will impact upon all 
TAFE Queensland Institutes and potentially the majority of instructional/ assessment 
staff and students. However, the scope of the pilot, at present, is limited to six 
qualifications across nine TAFE Queensland Institutes. Qualifications were selected 
to cover both large and small student populations, and institutes were selected to 
cover both metropolitan and country TAFE learners. Endorsement by the IRG 
(Product) for the inclusion of these participants in the pilot was to inform the 
development of the PLA system and formulate policy guidelines prior to more 
widespread implementation*. 

Prior to commencement of the pilot, a number of professional development activities 
were conducted for instruction/ assessment of pilot participants. These activities 
included an information dissemination teleconference and a one-day workshop for 
each industry area involved in the pilot. The workshops supplied participants with 
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the necessary information to understand PLA and to assist them in developing 
processes to implement the system with their students. 

North Point and Cooloola Sunshine Institutes of TAFE research 
partnership 

North Point Institute of TAFE (NPIT) is located in the northern growth corridor of 
Brisbane. NPIT is one of five Institutes in Queensland piloting PLA in their Diploma 
of Hospitality Management since mid-2000. Students in this Diploma include full 
and part-time students, including teenagers and mature aged students. Initially, the 
hospitality teaching team at NPIT met to discuss their approach to the 
implementation of PLA. A major issue identified was that of consistency in 
interpretation of performance criteria between teachers within the Institute and 
across other Institutes. North Point hospitality teachers approached Cooloola 
Sunshine Institute of TAFE (CSIT) hospitality teachers concerning the issues of 
consistency in assessing using PLA. It was decided that teachers from similar 
delivery/ assessment areas should meet. A meeting was arranged between the food 
and beverage teachers of each Institute. 

At the meeting between NPIT and CSIT food and beverage teachers, objective criteria 
were developed against each performance criteria as shown in Figure 1. For instance, 
the teachers felt that if they were to assess reliability and responsibility, the criteria 
they would evaluate would be attendance, handing in assignments and punctuality. 
On completion of these criteria it was felt that although they had been designed by 
food and beverage teachers, they were generic enough to translate into other 
assessment areas. Opinions of teachers in other areas confirmed this belief. 
Consequently, a template was designed stating the agreed criteria. Both institutes 
trialled this system throughout semester II, 2000. 

In January 2001, another meeting was held between teachers of NPIT and CSIT. This 
meeting was attended by teachers from all assessment areas of the Diploma of 
Hospitality Management from both institutes. At this meeting, teachers discussed the 
application throughout semester II, 2000 of the template which had been designed in 
the initial meeting. Three major issues emerged: 

1. the objective criteria developed were open to subjective interpretation 

2. the use of performance indicators 

3. the adequacy of the previously agreed objective criteria for assessment. 

The group felt that the objective criteria, which had been developed, were still open 
to subjective interpretation, thus jeopardising consistency. To overcome this, the 
group felt that a further column ought to be created on the template to clarify the 
objective criteria. For example, if a teacher uses the criteria attendance, what must a 
student achieve in relation to attendance to be awarded a credit or a distinction? 
Through further discussion, it emerged that this criteria may alter depending upon 
the unit of competency being assessed. Consequently, it was decided that the criteria 
to be included in the additional column should be specified by individual teachers, 
for their particular assessment area. However, where more than one teacher assesses 
a unit of competency or cluster, these teachers should agree on the criteria. 



Figure 1: Levels of performance 



Student name 




Qualification 


Group/class 


Unit code 


Unit title 


Performance description 

1 Demonstrate: 


Criteria PF/ Comments 

Ind 


Credit ... advanced reliability 
and responsibility 


Attendance 3 


Distinction ... exceptional 


Handing in assignments 4 


reliability and responsibility 


Punctuality 5 


2 Be able to: 

Credit ... access additional 
relevant information 


Assignment standard ^ 




Research skills 4 


Distinction ... acquire and apply 
additional information and use 
effectively and efficiently 


Liaison with relevant 5 

persons in the workplace 


3. Present: 

Credit ...work with originality 
and creativity 


Assignment presentation 3 


Distinction ... significant quality 


Practical application 4 


work output with creativity and 
originality 

4. Demonstrate: 


Selling, greeting, etc 5 


Credit ... positive approaches to 
learning 


Class participation 


Distinction ... positive 


Attitude and aptitude 


approaches to learning with high 
degree of motivation 

5. Be able to: 


Motivation 


Credit ... work independently 
and / or within a team 


Teamwork 3 


Distinction ... display a 


Self-management 4 


significant level of 
independence 


Using initiative 5 



5 



6. Provide evidence of: 



Credit ... planning and resource 
allocation to ensure completion 


Personal organisation 


3 


of tasks 


Time planning 


4 


Distinction : ... monitoring and 
evaluating work and identifying 
opportunities for improvement 


Resource utilisation 


5 


7. Demonstrate: 


Distinction ... mastery of tasks 


Exhibits a level of 5 in 


Y 


ensuring flexibility efficiency 


all aims 1-6 above with 




and effectiveness 


consistency throughout 
the unit 


or 

N 



Source: NPIT and CSIT meeting 1. 



1. The use of performance indicators 

Most teachers agreed that it is confusing using numerical performance indicators for 
formative assessment and then finally awarding a level of performance achieved as a 
credit or a distinction. Consequently, consensus was reached to make the formative 
performance indicators J for a competency, C for a credit and D for a distinction - the 
same as the final levels of performance awarded. However, at this stage, it was 
stressed that these performance indicators are precisely this. They are awarded to 
give the student an indication of the level at which they are performing, in their 
various assessment items, towards a competency or performance level. 



2. The adequacy of the previously agreed objective performance criteria for 
assessment 

The teachers reviewed the individual objective performance criteria developed by 
the food and beverage teachers for each level of performance descriptor and made 
minor changes as shown in Figure 2. For example, under the criteria '... access 
additional relevant information/ ... acquire and apply additional information and 
use effectively and efficiently', the teachers added a performance criteria of 
'discussion/ contribution'. 

Finally, a new template was developed to reflect the changes referred to above. This 
template, shown in Figure 2, will be trailed at both NPIT and CSIT throughout 
semester I, 2001. 
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Figure 2: Levels of performance 



Student Name 



Qualification 


Group/class 






Unit code 


Unit title 






Performance description 


Suggested 


Teacher criteria Pf/ 


Comments 




criteria 


Ind 




1 Demonstrate 








Credit ... advanced reliability 
and responsibility 


Attendance 


J 




Distinction ... exceptional 


Handing in 






reliability and responsibility 


assignments 


t 






Punctuality 


D 




2 Be able to: 








Credit ... access additional 
relevant information 


Assignment 

standard 


J 




Distinction ... acquire and apply 


Research skills 






additional information and use 


Liaison with 
relevant 


t 




effectively and efficiently 


D 




persons in the 


workplace 


Discussion/ 




contribution 






3. Present: 








Credit ...work with originality 


Assignment 


J 




and creativity 


presentation 
and content 






Distinction ... significant quality 
work output with creativity and 


Practical 


C 




originality 


application 








Communication 

skills 


D 




4. Demonstrate: 








Credit ... positive approaches to 


Class 


J 




learning 


participation 






Distinction ... positive 
approaches to learning with high 
degree of motivation 


Attitude and 
aptitude 


C 




Motivation 


D 
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5. Be able to: 



Credit ... work independently 
and / or within a team 

Distinction . . . display a 
significant level of 
independence 



Teamwork 

Self- 

management 

Using initiative 



J 

C 

D 



6. Provide evidence of: 



Credit ... planning and resource 


Personal 


J 


allocation to ensure completion 


organisation 




of tasks 


Time planning 


C 


Distinction: ... monitoring and 




D 


evaluation of work and identify 


Resource 




opportunities for improvement 
7. Demonstrate: 


utilisation 

Analysis skills 

Self-reflection / 
Initial reflection 

Identification of 

improvement 

opportunities 




Distinction ... mastery of tasks 


Exhibits a level 


Y 


ensuring flexibility efficiency 


of 5 in all 


I 


and effectiveness 


relevant aims in 
1 - 6 above 


or 




with 

consistency 
throughout the 
unit 


N 



Source: NPIT and CSIT meeting 2. 



Conclusions 

This paper describes how NPIT and CSIT are using a partnership arrangement to 
conduct ongoing research and review aimed at improving the quality of the new 
PLA system being piloted in Queensland. It explains how teachers, as practitioners, 
are using action research to develop a consistent, generic, quality approach to PLA, 
which in turn may be translated for use with other Training Packages. 

To date, as a result of this ongoing action research, consistency has been developed 
on two levels: 

1. Across institutes 

2. Within institutes. 




1. Across institutes 

Across institutes, consistency has been developed by: 

1. using agreed performance criteria 

2. using agreed performance indicators. 



2. Within institutes 

Within institutes, consistency has been developed by: 

1. using agreed performance criteria 

2. using agreed performance indicators 

3. using agreed individual unit of competency cluster criteria. 



Perhaps the next stage will be to develop a closer alignment of the institutes 
regarding clustering arrangements for assessment purposes. This, in turn, would 
develop the opportunity to create agreed criteria at the individual unit of 
competency/ cluster levels across institutes. If this can be achieved, it could form the 
basis of a consistency model which could be replicated across other institutes, other 
qualifications and other disciplines. 



Note 

*In December 2000 a full report on the effectiveness of the PLA system was 
completed and presented to the working party for consideration (Sanderson 2000). 
Based on this report, the Board of TAFE Queensland endorsed the continuation of 
the pilot for the year 2001 pending consideration of this report, including the 
recommendations by the working party. 
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